System for providing a metaverse-based virtualized image and method therefor

ABSTRACT

The present disclosure provides a method of providing a metaverse-based virtualized image, the method including capturing an image in real time via an IP camera, storing the captured image, recognizing at least one face and object from the captured image, virtualizing the recognized at least one face and object, and outputting a virtualized image information including a virtualized character and a virtual object.

CROSS REFERENCE TO RELATED APPLICATIONS

Pursuant to 35 U.S.C. § 119(a), this application claims the benefit ofan earlier filing date and right of priority to InternationalApplication No. PCT/KR2021/019460, filed on Dec. 21, 2021, the contentsof which are hereby incorporated by reference herein in its entirety.

BACKGROUND OF THE DISCLOSURE Field of the Disclosure

The present disclosure relates to a system for providing ametaverse-based virtualized image and method therefor.

Discussion of the Related Art

Recently, virtual reality, augmented reality, and mixed realitytechnologies that apply computer graphics technology have beendeveloped. At this time, virtual reality technology refers to atechnology that uses a computer to build a virtual space that does notexist in the real world and makes it feel real, and augmented reality ormixed reality technology refers to a technology that expressesinformation generated by a computer over the real world, i.e., atechnology that combines the real world and the virtual world togetherto enable interaction with a user in real time.

Among them, augmented reality and mixed reality technologies are used incombination with technologies in various fields (e.g., broadcastingtechnology, medical technology, and game technology). Typical examplesof the use of augmented reality technology in the field of broadcastingtechnology includes a case that a weather map in front of a weatherforecaster on TV changes naturally and a case that an advertisementimage that does not exist in the stadium is inserted on the screen andbroadcasted as if existing in the stadium actually. In particular, theseaugmented reality and mixed reality technologies have been implementedand provided as various application services with the advent ofsmartphones.

Metaverse is a representative service that provides augmented reality ormixed reality to users. Metaverse is a combination of ‘meta’, whichmeans virtual, and ‘universe’, which means real world, and refers to athree-dimensional virtual world where social/economic/culturalactivities take place like the real world. In metaverse, users createtheir own avatars, communicate with other users, and engage in economicactivities, and thus everyday life can be implemented in the virtualworld.

In addition, an Internet Protocol (IP) camera is a sort of a digitalcamera that is generally disposed for surveillance in general, andunlike analog Closed-Circuit Television (CCTV) cameras, corresponds to acamera that can receive data through computer networks and Internet.Most cameras that do such a job are web-cams, but the IP cameragenerally corresponds to only surveillance purposes, which can beaccessed directly over a network connection.

However, when using an IP camera, data is transmitted and receivedthrough Internet, so there is always a problem of privacy invasion thatwhen and how images taken by the IP camera will be released to theoutside.

In the real situation where the number of single-person households,double-income households, and elderly households is increasing, it isexpected that measures to solve the problem of privacy invasion and useIP cameras will be needed.

SUMMARY OF THE DISCLOSURE

Accordingly, embodiments of the present disclosure are directed to asystem for providing a metaverse-based virtualized image and methodtherefor that substantially obviate one or more problems due tolimitations and disadvantages of the related art.

One object of the present disclosure is to provide a system forproviding a metaverse-based virtualized image and method therefor.

Objects obtainable from the present disclosure are non-limited by theabove-mentioned object. And, other unmentioned technical objects can beclearly understood from the following description by those havingordinary skill in the technical field to which the present disclosurepertains.

Additional advantages, objects, and features of the disclosure will beset forth in the disclosure herein as well as the accompanying drawings.Such aspects may also be appreciated by those skilled in the art basedon the disclosure herein.

To achieve these objects and other advantages and in accordance with thepurpose of the invention, as embodied and broadly described herein, amethod of providing a metaverse-based virtualized image may includecapturing an image in real time via an IP camera, storing the capturedimage, recognizing at least one face and object from the captured image,virtualizing the recognized at least one face and object, and outputtinga virtualized image information including a virtualized character and avirtual object.

The recognizing the at least one face and object from the captured imagemay include requesting an analysis of the image by transmitting frameand sound information of the captured image to a cloud server based on apreset period and detecting a change of the recognized face and object.

The cloud server may recognize a face and object of the captured imagebased on an AI algorithm, and the AI algorithm may include at least oneof object detection, color detection, face identification, objecttracking, action detection, or sound analysis.

The virtualizing the recognized at least one face and object may furtherinclude 3D-modeling the recognized face into a character and 3D-modelingthe object into a virtual object.

The method may include generating a virtualized image informationincluding the character and the virtual object, and locations of thecharacter and the virtual object in the virtualized image informationare equal to locations of the recognized face and object in the capturedimage.

The method may further include saving a first virtualized imageinformation including a first character and a first virtual object tothe cloud server and sharing by the cloud server the first virtualizedimage information with a terminal.

The terminal may generate and output a metaverse image based on thevirtualized image information and a virtualized image in the terminal.

The method may include detecting a preset situation and providing anotification if the preset situation is detected.

Based on detecting the preset situation, an original copy of thecaptured image may be saved to the cloud server.

The preset situation may include a case that at least one of anunregistered character or a virtual object is recognized from thevirtualized image information, a case that a registered character isrecognized from the virtualized image information, and a case that theregistered character does not move over a preset time in the virtualizedimage information.

The method may further include receiving a second virtualized imageinformation including a second virtual object from the cloud server,outputting the second virtualized image information, receiving a signalof selecting the second virtual object, searching for an object relatedto the second virtual object or an object similar to the second virtualobject, and outputting the found object related to the second virtualobject or the similar object.

The captured image may be stored in a memory within the IP camera.

The preset period may be determined based on at least one of therecognized face or the recognized object.

In another aspect of the present invention, as embodied and broadlydescribed herein, a system for providing a metaverse-based virtualizedimage may include an IP camera capturing an image in real time, a cloudserver configured to receive frame and sound information of the capturedimage, recognize at least one face and object from the captured image,virtualize the recognized at least one face and object, and generate avirtualized image information including the virtualized face and object,and a terminal configured to receive the virtualized image informationfrom the cloud server and output the received virtualized imageinformation, wherein the cloud server may recognizes the at least oneface and object based on an AI algorithm.

The effects of a metaverse-based virtualized image providing method andsystem according to the present disclosure are described below.

According to at least one of embodiments of the present disclosure,there is an advantage in that privacy invasion anxiety may be eliminatedwhen using a home CCTV.

According to at least one of embodiments of the present disclosure,there is an advantage in that emotional and fun elements may be extendedthrough new virtual space creation and enjoyable interaction betweenfamily members.

According to at least one of embodiments of the present disclosure, newhome appliances, furniture, and interior accessories are applied to avirtual space, thereby providing convenience of interior-relatedshopping.

Further scope of applicability of the present invention will becomeapparent from the detailed description given hereinafter.

However, it should be understood that the detailed description andspecific examples, while indicating preferred embodiments of theinvention, are given by illustration only, since various changes andmodifications within the spirit and scope of the invention will becomeapparent to those skilled in the art from this detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the disclosure and are incorporated in and constitute apart of this application, illustrate embodiment(s) of the disclosure andtogether with the description serve to explain the principle of thedisclosure. The above and other aspects, features, and advantages of thepresent disclosure will become more apparent upon consideration of thefollowing description of preferred embodiments, taken in conjunctionwith the accompanying drawing figures. In the drawings:

FIG. 1 is a diagram illustrating an embodiment of outputting an imagecaptured by an IP camera of the related art;

FIG. 2 is a diagram illustrating an example of outputting an imagecaptured by an IP camera according to one embodiment of the presentdisclosure;

FIG. 3 is a diagram illustrating operations of an IP camera and a cloudserver according to one embodiment of the present disclosure;

FIG. 4 is a diagram illustrating a result from analyzing an imagethrough an AI algorithm of a cloud server according to one embodiment ofthe present disclosure;

FIG. 5 is a diagram illustrating an example of virtualizing an image ina cloud server according to one embodiment of the present disclosure;

FIG. 6 is a diagram illustrating an example of detecting a presetsituation and providing a notification to a terminal by a cloud serveraccording to one embodiment of the present disclosure;

FIG. 7 is a diagram illustrating an example of performing objecttracking in a cloud server according to one embodiment of the presentdisclosure;

FIGS. 8 to 12 are diagrams illustrating schemes of utilizing virtualizedimage information according to one embodiment of the present disclosure;and

FIG. 13 is a flowchart illustrating a method of providing ametaverse-based virtualized image according to one embodiment of thepresent disclosure.

DETAILED DESCRIPTION OF THE DISCLOSURE

Description will now be given in detail according to exemplaryembodiments disclosed herein, with reference to the accompanyingdrawings. For the sake of brief description with reference to thedrawings, the same or equivalent components may be provided with thesame reference numbers, and description thereof will not be repeated. Ingeneral, a suffix such as “module” and “unit” may be used to refer toelements or components. Use of such a suffix herein is merely intendedto facilitate description of the specification, and the suffix itself isnot intended to give any special meaning or function. In the presentdisclosure, that which is well-known to one of ordinary skill in therelevant art has generally been omitted for the sake of brevity. Theaccompanying drawings are used to help easily understand varioustechnical features and it should be understood that the embodimentspresented herein are not limited by the accompanying drawings. As such,the present disclosure should be construed to extend to any alterations,equivalents and substitutes in addition to those which are particularlyset out in the accompanying drawings.

It will be understood that although the terms first, second, etc. may beused herein to describe various elements, these elements should not belimited by these terms. These terms are generally only used todistinguish one element from another.

It will be understood that when an element is referred to as being“connected with” another element, the element can be directly connectedwith the other element or intervening elements may also be present. Incontrast, when an element is referred to as being “directly connectedwith” another element, there are no intervening elements present.

A singular representation may include a plural representation unless itrepresents a definitely different meaning from the context.

In the present application, terms such as “include” or “has” are usedherein and should be understood that they are intended to indicate anexistence of several components, functions or steps, disclosed in thespecification, and it is also understood that greater or fewercomponents, functions, or steps may likewise be utilized.

With the increase in double-income families, needs for the use of CCTVsto monitor the conditions of children, pets, or elderly parents aresteadily increasing. Regarding an IP camera mainly used for a CCTV, animage is taken with an IP camera, the taken image is stored in a cloudserver, etc., and both real-time and stored images of the IP camera canbe checked with applications. However, there is still a risk of use dueto privacy invasion between family members and IP camera hacking issue.

In addition, by analyzing an images from an IP camera, it is difficultto alarm with good performance. If sound or motion is detected, an alarmis given, but there is also a problem that alarms are generated toofrequently but an important alarm is not given.

In particular, what a user wants to know through CCTV is whether a childwent to kindergarten or academy on time, whether a stranger came home,whether parents are active enough, or what a pet is doing, which is thepurposes of the CCCTV, but the user doesn't have to see a real house andreal family members' faces.

Accordingly, the present disclosure intends to provide a method ofvirtualizing and displaying a CCTV image instead of an actual image.That is, an image may be captured through an IP camera and an originalimage may be stored in a memory of the IP camera. More specifically, inan embodiment of the present disclosure, an indoor metaverse environmentmay be constructed using face, motion, and object recognitiontechnology, and a family member may be 3D modeled as a character. Thiswill be described in detail with reference to the accompanying drawings.

FIG. 1 is a diagram illustrating an embodiment of outputting an imagecaptured by an IP camera of the related art.

Referring to FIG. 1 , an IP camera 101 may capture an image 102 in realtime. In doing so, the image 102 captured by the IP camera 101 may beexternally transmitted via a network. For example, a terminal 200 mayreceive the image 102 captured by the IP camera 101 and output itthrough a display.

In this case, when the image 102 captured by the IP camera 101 isdirectly transmitted to the terminal from the IP camera 101 via thenetwork, there may be a risk that the captured image 102 may remain inthe network. Namely, That is, in the case of the IP camera 101, it iscommon to be installed in the home for monitoring purposes, but since animage in the home is transmitted through the network to be transmittedto the terminal 200, there is a risk of hacking, which may lead toprivacy invasion. In order to compensate for this point, an embodimentof the present disclosure proposes a method of virtualizing a face andan object recognized from the image 102 captured by the IP camera 101and providing the same to the terminal 200.

FIG. 2 is a diagram illustrating an example of outputting an imagecaptured by an IP camera according to one embodiment of the presentdisclosure.

Referring to FIG. 2 , an IP camera 101 may capture an image 102 in realtime. In this case, the captured image 102 may be stored in a memory 103of the IP camera 101. Here, the memory 103 may include an SD card andthe like.

In one embodiment of the present disclosure, the captured image 102 maybe transmitted and virtualized via a cloud server. Thereafter, havingreceived the virtualized image 201 via the cloud server, a terminal 200may output the virtualized image 201. In this case, the terminal 200 maycorrespond to a metaverse client terminal configured to providemetaverse based information. The terminal 200 may include one of amobile phone, a projector, a smartphone, a notebook computer (e.g., alaptop computer), a digital broadcast terminal, Personal DigitalAssistants (PDA), Portable Multimedia Player (PMP), a navigation device,a slate PC, a tablet PC, an ultrabook, a wearable device (e.g., awatch-type terminal (smartwatch), a glasses-type terminal (smartglasses), a head mounted display (HMD)), etc.

Hereinafter, how the cloud server stores the captured image 102 and howthe terminal 200 outputs the virtualized image 201 will be described indetail. This will be described in detail with reference to FIG. 3 andFIG. 4 .

Accordingly, since an original copy of the captured image 102 is storedin the memory 103 of the IP camera 101 and the image 201 virtualized viathe cloud server is outputted from the terminal 200, privacy can beprotected.

FIG. 3 is a diagram illustrating operations of an IP camera and a cloudserver according to one embodiment of the present disclosure.

Referring to FIG. 3 , an IP camera 101 may include a scene changedetection 104, a sound extractor 105 and a server interface 106. Inaddition, a cloud server 300 may include an API gateway 301, an AIalgorithm 302 and a cloud server memory 303. Since the components shownin FIG. 3 are not essential for implementation of the IP Camera 101 andthe cloud server 300, the IP camera 101 and the cloud server 300described in the present specification may have components more or lessthan the above-listed components. In addition, all of the aforementionedcomponents are not shown in detail in the accompanying drawings, andonly some important components may be shown in the accompanyingdrawings. However, although not all of them are shown, it isunderstandable to those skilled in the art that at least the componentsof FIG. 3 may be included in the IP camera 101 and the cloud server 300to implement functions.

The IP camera 101 may capture an image in real time.

The memory 103 may store the image captured in real time by the IPcamera 101. In this case, the memory 103 may be built in the IP camera103 and may include an SD card. A user may then check the image storedin the memory 103.

The scene change detection 104 may extract frame information of theimage captured by the IP camera 101. The scene change detection 104 maytransmit the extracted frame information to the cloud server 300 via theserver interface 106. The scene change detection 104 may detect a scenechange of the image captured in real time by the IP camera 101. In doingso, the scene change detection 104 may directly detect a scene change orset up an alarm period of a scene change based on information (includingperiod information for example) received from the cloud server 300described later.

The sound extractor 105 may extract sound information of the imagecaptured by the IP camera 101. The sound extractor 105 may transmit theextracted sound information to the cloud server 300 via the serverinterface 106.

The server interface 106 may transmit frame (e.g., 1 frame per 5seconds) and sound information of the image captured by the IP camera101 to the cloud server 300 and make a request for an analysis of theimage.

The APIR gateway 301 may forward the request for the analysis of theframe information and the sound information received via the serverinterface 106 of the IP camera 101 to the AI algorithm 302 of the cloudserver 300.

The AI algorithm 302 may perform at least one of object detection, colordetection, face identification, object tracking, action detection andsound analysis on the received frame information and the received soundinformation. In this case, the above-described operation performed bythe AI algorithm 302 is for a currently constructed algorithm, and moreaccurate information expression may be possible when other algorithmssuch as facial expression analysis and the like are added.

The AI algorithm 302 may store the results of object detection, colordetection, face identification, object tracking, action detection, andsound analysis performed on the received frame information and soundinformation in the cloud server memory 303.

In addition, the cloud server 300 may save the frame information andsound information received from the IP camera 101 to the cloud servermemory 303.

FIG. 4 is a diagram illustrating a result from analyzing an imagethrough an AI algorithm of a cloud server according to one embodiment ofthe present disclosure.

An image 102 captured via the aforementioned IP camera (not shown) maybe analyzed as shown in FIG. 4 .

Specifically, an AI algorithm in a cloud server may recognize at leastone object in the image 102. In addition, if the recognized object is aperson, the AI algorithm may recognize a face 107 of the person. In thiscase, if the face 107 corresponds to a registered user, the AI algorithmmay identify the face 107 of the image 102. For example, the A algorithmmay identify that the face 107 of the image 102 is ‘our daughter’.

In addition, the AI algorithm may specifically recognize at least oneobject in the image 102. For example, the AI algorithm may identify asofa, a table, a flowerpot, an air purifier, a curtain, and the like inthe image 102.

In addition, the AI algorithm may recognize the color and arrangement ofthe recognized at least one object. For example, the AI algorithm mayrecognize the recognized object as ivory for the sofa, beige for thetable, gray for the flowerpot, white for the air purifier, and khaki forthe curtain. In addition, the AI algorithm may recognize a location,size, ratio and the like occupied by the recognized object in space.

The recognized image information may be virtualized as shown in FIG. 5 .

FIG. 5 is a diagram illustrating an example of virtualizing an image ina cloud server according to one embodiment of the present disclosure.

Referring to FIG. 5 , a cloud server 300 may virtualize a capturedimage. Specifically, the cloud server 300 may perform virtualization tomatch a location and a color tone of an object based on imageinformation analyzed like FIG. 4 .

In one embodiment of the present disclosure, the cloud server 300 may3D-model a recognized face as a character 304 and also 3D-model arecognized object as a virtual object. In this case, the character 304may be a preconfigured metaverse character 304 or may correspond to ametaverse character 304 similar to the recognized face in case ofabsence of the preconfigured metaverse character 304.

In addition, the virtual object may correspond to a virtual objectsimilar to the recognized object in size and color. The character 304and the virtual object used herein may be extracted from the databasestored in the cloud server 300.

As another embodiment, the character 304 and virtual object used hereinmay be extracted from another external resource usable in the cloudserver 300.

In addition, when the cloud server 300 3D-models the recognized face asthe character 304, it may precisely represent the character 304 usinginformation such as action or face expression information and the like.

The cloud server 300 may generate virtualized image information 305including the character 304 and the virtual object. In doing so, thecloud server 300 may generate the virtualized image information 305 in amanner of disposing locations and sizes of the 3D-modeled character 304and the 3D-modeled virtual object to match the locations and sizes of anoriginal image (i.e., raw data). Thereafter, the cloud server 300 maysave the generated virtualized image information 305 to a cloud servermemory 303.

In one embodiment of the present disclosure, a terminal 200 may receive(or read) the virtualized image information 305 and sound informationfrom the cloud server 300. In addition, the sound information may betransmitted/received and stored in a manner of being included in orseparated from the virtualized image information 305.

In one embodiment of the present disclosure, the terminal 200 mayreceive the virtualized image information 305 and sound information fromthe cloud server 300 via a server interface 201 of the terminal 200.

The terminal 200 may generate a metaverse image 203 using the receivedvirtualized image information 305 and sound information and avirtualized image 202 in the terminal 200. In this case, the virtualizedimage 202 may include an image brought from an external server of theterminal 200 or correspond to an image stored in a resource within theterminal 200. Namely, the terminal 200 may receive only the virtualizedimage information 304 and sound information from the cloud server 300and may render the virtualized image 202 stored in the resource.Thereafter, the terminal may output a metaverse image 203 through adisplay.

Accordingly, as the virtualization-processed character 304 and virtualobject are stored in the cloud server 300, the problem of privacyinvasion can be solved despite using the IP camera 101.

FIG. 6 is a diagram illustrating an example of detecting a presetsituation and providing a notification to a terminal by a cloud serveraccording to one embodiment of the present disclosure;

Referring to FIG. 6 , a cloud server 300 may detect a preset situationbased on virtualized image information 305 stored in a cloud servermemory 303.

In one embodiment of the present disclosure, the cloud server 300 mayprovide a notification to a terminal 200 via an alert manager 306 incase of detecting a preset situation. Here, the preset situation mayinclude a case that at least one of an unregistered character 307 or avirtual object is recognized from a virtualized image information 305, acase that a registered character 304 is recognized from the virtualizedimage information 305, or a case that the registered character 304 doesnot move over a preset time in the virtualized image information 305.For one example, the cloud server 300 may detect the case that theunregistered character 307 is recognized from the virtualized imageinformation 305. For another example, the cloud server 300 may detectthe case that t the registered character 304 does not move over a presettime in the virtualized image information 305. For further example, thepreset situation may include an abnormal sound information (e.g., childcrying sound), a fall (activity detection), etc.

In case of detecting the preset situation, the cloud server 300 may savean original copy of a captured image to a cloud server memory 303. Inaddition, in case of detecting the preset situation, the cloud server300 may save the virtualized image information 305 and an original image(i.e., raw data) of the unregistered character 306 to the cloud servermemory 303.

The terminal 200 may receive a notification from the cloud server 300via a server interface 201. The terminal 200 may output the notificationto a user via an alert manager 204 of the terminal 200. For example, theterminal 200 may output an original image (i.e., raw data) of theunregistered character 307 to a popup window 205 via a display. In doingso, the server interface 201, the alert manager 204, the popup window205 and the like may operate in a metaverse related application 206installed on the terminal 200.

In addition, the cloud server 300 may determine a preset situation invarious ways if more AI algorithms are present in the cloud server 300.Namely, since the cloud server 300 determines a preset situation usingan AI algorithm based on the captured image 102 and the virtualizedimage information 305, the cloud server 300 may determine a presetsituation in more various ways if more AI algorithms are provided.

Through the above-described embodiment, a user normally checks thevirtualized image information 305 via the terminal 200 only. If a presetsituation occurs, the user may check an original image (raw data) of theunregistered character 307 via the terminal 200.

Through this, the user can usually be safe from the risk of privacyinvasion or hacking, and can check the original image (raw data) in anemergency situation.

FIG. 7 is a diagram illustrating an example of performing objecttracking in a cloud server according to one embodiment of the presentdisclosure;

An IP camera 101 may make a request for an analysis of frame informationof a captured image 102 via an API gateway 301 of a cloud server 300. Inaddition, the IP camera 101 may transmit the frame information of thecaptured image 102 to a cloud server memory 303.

The cloud server 300 may perform an object tracking 308 that is one ofAI algorithms 302. Namely, the cloud server 300 may perform tracking onan object recognized from a virtualized image information 305 generatedfrom virtualizing the captured image 102.

In one embodiment of the present disclosure, a preset period may bedetermined based on at least one of a recognized face and an object.

Specifically, if there is a moving object such as a person or an animalin the captured image 102, the IP camera 101 may set up a short analysisrequest period (e.g., 1 frame per second). If there is no moving object,the IP camera 101 may set up a long analysis request period (e.g., 1frame per second). Namely, the IP camera 101 may perform recognition ofa simple object. If there are many motions of an object, the IP camera101 may transmit frame information to the cloud server 300 morefrequently and make a request for an image analysis.

The cloud server 300 may save the virtualized image information 305 on amoving object to the cloud server memory 303. Thereafter, the cloudserver 303 may transmit the virtualized image information 305 to aterminal (not shown). Accordingly, the terminal may receive thevirtualized image information 305 on which the object tracking has beenperformed. The terminal may output the virtualized image information 305in which a character moves naturally.

FIGS. 8 to 12 are diagrams illustrating schemes of utilizing virtualizedimage information according to one embodiment of the present disclosure.

Through the above-described method, the image 102 captured via the IPcamera 101 may be virtualized via the cloud server 300. The virtualizedimage information 305 is shared externally and may be outputted throughthe terminal 200. The virtualized image information 305 does not show anactual face or household objects as they are, and uses characters andvirtual objects, so there is no concern about invasion of privacy evenif they are disclosed externally. Accordingly, various embodiments inwhich the virtualized image information 305 may be used will bedescribed with reference to FIGS. 8 to 12 .

Referring to FIG. 8 , in case of using a metaverse-based virtualizedimage providing system and method, a virtual door to another virtualspace is included in the virtualized image information 305, wherebyconnection to another virtual space is available.

Specifically, FIG. 8 (a) shows a first virtualized image information801, FIG. 8 (b) shows a second virtualized image information 802, andFIG. 8 (c) shows a third virtualized image information 803. FIGS. 8 (a)to 8 (c) show embodiments of outputting virtualized image informationsfrom first to third terminals, respectively. Here, the first to thirdterminals may correspond to terminals registered as friends with eachother via the cloud server 300 or a metaverse related application.Alternatively, the first to third terminals may correspond to terminalshaving authentication keys capable of visiting virtualized imageinformations 801 to 803 to each other, respectively.

In this case, the first virtualized image information 801 may include avirtual door 801 a connecting to a virtual space. Here, the virtualspace may include a virtual space in which on-line meeting is availableby inviting friends or family members, a game space capable of enjoyinga game with friends or family members, and a virtual space capable ofenjoying activities with friends or family members.

Accordingly, users may have meetings and games through virtual spaceswithout meeting each other actually. In addition, the users may use avirtual object virtualized based on an object actually captured via theIP camera 101. In addition, the virtual door 801 a/802 a/803 a includedin the virtualized image information 801/802/803 may correspond to anobject connectible to another virtual space despite being a virtualobject that does not exist actually. Users may have or enjoy a meeting,a game and the like in a new virtual space by being connected to thecloud server 300 via the virtual doors 801 a, 802 a and 803 a.

Referring to FIG. 9 , the terminal 200 may output a virtualized imageinformation 901. In this case, the virtualized image information 901 maycorrespond to a virtualized image information 901 on an image capturedby a user of the terminal 200 or another user's virtualized imageinformation 901 received via the cloud server 300.

In doing so, a signal 903 of selecting a virtual object 902 included inthe virtualized image information 901 may be received. The terminal 200may search for an actual object corresponding to the virtual object 902or an actual object similar to the virtual object 902 via the cloudserve r 300. Thereafter, the terminal 200 may output a search result904. Here, the search result 904 may include information on the actualobject corresponding to the virtual object 902 or the actual objectsimilar to the virtual object 902.

The terminal 200 may receive a signal 906 of selecting a first object905 from the search result 904. Hence, the terminal 200 may place thefirst object 905 in the virtualized image information 901.

In addition, although FIG. 9 illustrates the embodiment of placing thefirst object 905 in the virtualized image information 901, an embodimentof purchasing the first object 905 according to the search result 904may be taken into consideration as well. Likewise, as the terminal 200receives the signal 906 of selecting the first object 905 from thesearch result 904, it may output a site at which the first object 905can be purchased.

FIG. 10 (a) illustrates an embodiment of creating a metaverse characterbased on an actual face, and FIG. 10 (b) illustrates an embodiment ofsharing a virtualized space and character.

As shown in FIG. 10 (a), a metaverse character 1002 may be created basedon an actually captured image 1001. A method of creating the metaversecharacter 1002 is as described above.

As shown in FIG. 10 (b), a virtualized character and a virtual objectmay be shared with other persons. Namely, as described above, users mayshare the virtualized image information 305 including the virtualcharacter and the virtual object via the cloud server 300. Through this,each user may place the virtual object in a virtualized space of itsown, include sound information therein, and visit a virtualized space ofanother user. In addition, the cloud server 300 may provide thevirtualized space with user information on a user visiting thevirtualized space and output the user information through the terminal200.

Namely, in case of using the metaverse-based virtualized image providingsystem and method, a platform of making a new influencer can beprovided.

Through this, many influencers can share their own homes, which cancreate new advertising effects. For example, various types ofadvertising effects may occur, including virtualized home appliances,furniture, interior accessories, character clothes, and accessories. Amethod of purchasing such a virtual object is as described above withreference to FIG. 9 .

When the metaverse-based virtualized image providing method according toone embodiment of the present disclosure is used, it is applicable notonly to an indoor environment but also to an outdoor environment.

FIG. 11 (a) is a diagram illustrating a general amusement park entranceline, a department store entrance line, and an interior of a restaurant,and FIG. 11 (b) is a diagram illustrating them as the virtualizationimage information 305.

Namely, a user is unable to know an internal situation until the useractually goes to an amusement park, a department store, a restaurant,etc. Only after arriving, the user is able to know how many people arein the amusement park, the department store, or the restaurant. On theother hand, when the metaverse-based virtualized image providing methodaccording to one embodiment of the present disclosure is used, the usermay be informed of the situation in advance through the virtualizedimage information 305. More specifically, the IP cameras 101 are alsorecently used in amusement parks, department stores, and restaurants. Inthis case, the cloud server 300 may change the image 102 captured by theIP camera 101 installed in a public facility such as an amusement park,a department store, a restaurant, or the like into the virtualized imageinformation 305. Since the faces of real people included in the capturedimage 102 have portrait rights, the amusement park, department store,restaurant, etc. cannot directly use the image 102. On the other hand,when the captured image 102 is changed into the virtualized imageinformation 305 via the cloud server 300, the public facility may sharethe virtualized image information 305 externally.

Accordingly, if the IP camera 101 is installed in a desired place, auser may immediately determine the situation in the desired placethrough the virtualized image information 305 corresponding to the image102 captured by the IP camera 101 without infringing on other people'sportrait rights.

In addition, using the metaverse-based virtualized image providingmethod according to one embodiment of the present disclosure, it is ableto provide a platform capable of providing event information andpromotion or to link with various AR/VR games. In FIG. 12 , a case ofproviding department store virtualized image information based on adepartment store in which an IP camera 101 is installed will bedescribed as an example. Yet, this is only an example, and of course, itcan be applied to every place where the IP camera 101 is installed.

FIG. 12 (a) is a diagram of providing event information in departmentstore virtualized image information, and FIG. 12 (b) is a diagram ofproviding promotion information in department store virtualized imageinformation. A user may output department store virtualized imageinformation through the terminal 200.

As shown in FIG. 12 (a), a department store virtualization imageinformation may include an event information 1201. In addition, thedepartment store virtualized image information may include a placeindicator 1202 for guiding the event information 1201.

As shown in FIG. 12 (b), a department store virtualized imageinformation may include a promotion information 1203. When receiving asignal (not shown) of selecting the promotion information 1203, theterminal 200 may make a request for a subsequent operation to the cloudserver 300. The terminal 200 may output a location information of thepromotion information 1203 or a specific schedule information 1204 ofthe promotion information 1203 based on the instruction of the cloudserver 300.

FIG. 13 is a flowchart illustrating a method of providing ametaverse-based virtualized image according to one embodiment of thepresent disclosure.

Referring to FIG. 13 , in a step S1301, an image may be captured in realtime via an IP camera.

In a step S1302, the captured image may be saved. In this case, thecaptured image may be saved to a memory in the IP camera.

In a step S1303, at least one face and object may be recognized from thecaptured image. In this case, after at least one object has beendetected, if the detected object is a person or animal, a face may bedetected.

In addition, in the step S1303, the IP camera may make a request for ananalysis of an image by transmitting frame and sound information of thecaptured image to a cloud server based on a preset period. In this case,the preset period may be determined based on at least one of therecognized face and the recognized object.

The cloud server may detect a change of the recognized face and objectin the captured image. In doing so, the cloud server may recognize aface and object of the captured image using an AI algorithm. Here, theAI algorithm may include at least one of object detection, colordetection, face identification, object tracking, action detection andsound analysis.

In a step S1304, the recognized at least one face and object may bevirtualized. In the step S1304, the recognized face can be 3D-modeledinto a character. In addition, the recognized object may be 3D-modeledinto a virtual object.

In the step S1304, a virtualized image information including thecharacter and the virtual object may be generated. In this case,locations of the character and virtual object in the virtualized imageinformation may be equal to those of the recognized face and object inthe captured image. The cloud server may save the generated virtualizedimage information to a cloud server memory. Thereafter, the cloud servermay share the saved virtualized image information with the terminal.

In a step S1305, the virtualized image information including thevirtualized character and the virtual object may be outputted.Specifically, the terminal may generate and output a metaverse imageusing the virtualized image information received from the cloud serverand the virtualized image in the terminal.

In addition, it is a matter of course that the embodiments through FIGS.2 to 12 may be implemented by the method as shown in FIG. 13 .

The present disclosure mentioned in the foregoing description can beimplemented in a program recorded medium as computer-readable codes. Thecomputer-readable media may include all kinds of recording devices inwhich data readable by a computer system are stored. Thecomputer-readable media may include HDD (Hard Disk Drive), SSD (SolidState Disk), SDD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tapes,floppy discs, optical data storage devices, and the like for example andalso include carrier-wave type implementations (e.g., transmission viaInternet). Further, the computer may include the controller 180 of theimage editing device. The foregoing embodiments are merely exemplary andare not to be considered as limiting the present disclosure. The presentteachings can be readily applied to other types of methods andapparatuses. Thus, it is intended that the present disclosure covers themodifications and variations of this invention that come within thescope of the appended claims and their equivalents.

What is claimed is:
 1. A method of providing a metaverse-basedvirtualized image, the method comprising: capturing an image in realtime via an internet protocol (IP) camera; storing the captured image;recognizing a face and an object from the captured image; generating avirtual character and a virtual object corresponding to a virtualizationof the recognized face and the object, respectively, based on thecaptured image; and outputting a virtualized image information includingthe virtualized character and the virtual object.
 2. The method of claim1, wherein recognizing the face and the object from the captured imagecomprises: requesting an analysis of the captured image by transmittingframe and sound information of the captured image to a cloud serverbased on a preset period; and detecting a change of the recognized faceand object.
 3. The method of claim 2, wherein the face and the object ofthe captured image are recognized by the cloud server based on anartificial intelligence (AI) algorithm, and wherein the AI algorithmincludes at least one of object detection, color detection, faceidentification, object tracking, action detection, or sound analysis. 4.The method of claim 3, wherein the virtual character and virtual objectare generated by 3D-modeling the recognized face and 3D-modeling therecognized object.
 5. The method of claim 4, wherein locations of thevirtual character and the virtual object in the virtualized imageinformation respectively correspond to locations of the recognized faceand recognized object in the captured image.
 6. The method of claim 5,further comprising saving the virtualized image information to the cloudserver, wherein the first virtualized image information is shared with aterminal by the cloud server.
 7. The method of claim 6, wherein ametaverse image is generated and output by the terminal based on thevirtualized image information and a virtualized image stored in theterminal.
 8. The method of claim 7, further comprising: detecting apreset situation; and providing a notification based on detection of thepreset situation.
 9. The method of claim 8, wherein based on detectionof the preset situation, an original copy of the captured image is savedto the cloud server.
 10. The method of claim 9, wherein the presetsituation includes: wherein at least one of an unregistered character ora virtual object is recognized from the virtualized image information,wherein a registered character is recognized from the virtualized imageinformation, or wherein the registered character does not move during apreset time based on the virtualized image information.
 11. The methodof claim 10, further comprising: receiving a second virtualized imageinformation including a second virtual object from the cloud server;outputting the second virtualized image information; receiving a signalof selecting the second virtual object; searching for an object relatedor similar to the second virtual object; and outputting the objectresulting from the search.
 12. The method of claim 1, wherein thecaptured image is stored in a memory of the IP camera.
 13. The method ofclaim 2, wherein the preset period is determined based on at least oneof the recognized face or the recognized object.
 14. A system forproviding a metaverse-based virtualized image, the system comprising: anIP camera configured to capture an image; a cloud server configured to:receive frame and sound information of the captured image, recognize aface and an object from the captured image based on an AI algorithm,generate a virtual character and a virtual object corresponding to avirtualization of the recognized face and the object, respectively,based on the captured image; and a terminal configured to receive thevirtualized image information from the cloud server and output thereceived virtualized image information.